Topological insulators and fractional quantum Hall effect on the ruby lattice 
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We study a tight-binding model on the two-dimensional ruby lattice. This lattice supports several 
types of first and second neighbor spin-dependent hopping parameters in an s-band model that 
preserves time-reversal symmetry. We discuss the phase diagram of this model for various values of 
the hopping parameters and filling fractions, and note an interesting competition between spin-orbit 
terms that individually would drive the system to a Z2 topological insulating phase. We also discuss 
a closely related spin-polarized model with only first and second neighbor hoppings and show that 
extremely flat bands with finite Chern numbers result, with a ratio of the band gap to the band 
width approximately 70. Such flat bands are an ideal platform to realize a fractional quantum 
Hall effect at appropriate filling fractions. The ruby lattice can be possibly engineered in optical 
lattices, and may open the door to studies of transitions between quantum spin liquids, topological 
insulators, and integer and fractional quantum Hall states. 

PACS numbers: 71.10.Fd,71.10.Pm,73.20.-r,73.43.-f 



I. INTRODUCTION 

Topological phases of matter have received a great 
deal of attention recently. In part, this is motivated 
by the fractional quantum Hall effect and its possible 
role as a platform for topological quantum computation.^ 
However, there is also a more general interest in phases 
of quantum many-particle systems that can exhibit re- 
sponses and other properties that are a consequence of 
global {i.e. topological) features that are not captured in 
a local order parameter. A key example of such a phase 
is the time-reversal invariant topological insulator. In 
contrast to most of the previously known systems that 
exhibited some topological property (the integer quan- 
tum Hall effect being a notable exception), topological 
insulators do not require electron-electron interactions, 
but they are stable to interactions of weak to moderate 
strength^i"— The weakly-interacting nature of topologi- 
cal insulators has enabled accurate predictionsiSr— for 
a wide range of two and three dimensional systems^"— 
and experiment has followed with confirming data in a 
large and rapidly growing number of instances. ^^^^ 

The salient feature of topological insulators is that 
their boundaries possess a topologically protected 
"metallic" state that is robust to disorder Both 
the two dimensional^^^^ and the three dimensional^"— 
boundaries have been shown to exhibit interesting re- 
sponses to perturbations. In this work we focus on a two 
dimensional tight-binding model with a single 5-orbital 
on each lattice site. We study the so-called "ruby" lat- 
tice shown in Figdl We find that it exhibits a complex 
phase diagram that includes topological insulators at a 
number of filling fractions. This lattice has earlier played 
an important role in the study of topological order in spin 
models^iSi 

Starting with the work of Kane and Mele^^^^^ on 
the honeycomb lattice (and key earlier precedents by 
Haldane^^ in a spinless version), such simple non- 
interacting lattice models have helped to develop our un- 



derstanding of topological insulators Besides the hon- 
eycomb lattice^'— a number of other two dimensional 
lattices have been shown to support a topological insula- 
tor phase, including the decorated honeycomb lattice^ 
the checkerboard lattice, the square-octagon lattice^ 
the kagome lattice, and others. It is now well ap- 
preciated that such non-interacting lattice models pos- 
sess topological features that commonly occur in inter- 
acting models without spin- orbit coupling at the mean- 
field leveL^i^— Evidently, such simple lattice models 
contain rather rich physics. 

An interesting related topic of study is the class of in- 
sulators with nearly flat bands that possess finite Chern 
numbers. With a finite Chern number, a partial filling 
of the flat bands can lead to a fractional quantum Hall 
effect. To date, only a few lattice models have been pro- 
posed which are expected to lead to a fractional quantum 
Hall effect. ^^^^ The relevant figure of merit in such mod- 
els is the ratio of the band gap to the bandwidth of the 
flat band with a finite Chern number. In the model we 
discuss in this paper, we find this ratio can be as high as 
70, which is among the largest in the models reported in 
the literature thus far. 

The remainder of the paper is organized as follows. In 
Sec.|II]we introduce our tight-binding model, and discuss 
the basic features of the energy bands as a function of 
the hopping parameters. In Sec. Illll we discuss the phase 
diagrams at different filling fractions, and in Sec. HVl we 
discuss a closely related spinless (spin polarized) model 
with nearly flat bands and finite Chern number. Our 
main conclusions are given in Sec. |Vl 



II. HAMILTONIAN AND BAND STRUCTURE 

We study the Hamiltonian 

H = Ho^Hso, (1) 



FIG. 1. Schematic of the ruby lattice and illustration of the 
nearest-neighbor hopping, t,ti (real) and t' ^t'l (complex), and 
the three types of "second-neighbor" spin-orbit coupling or 
hopping indicated by the dashed or dot dashed lines, t2, ts and 
t^r- (a) The spin-orbit coupling strength t2 within a hexagon, 
(b) The spin-orbit coupling strength ts within a pentagon 
composed of one triangle and one square. The hoppings t2 
and ts are present on all such bonds of the type shown that are 
consistent with the symmetry of the lattice, (c) The unit cell 
of the ruby lattice, (d) Schematic of the hopping parameters 
used to obtain a flat band with a finite Chern number and 
W/Eg ^ 70. (e) The first Brillouin zone of the ruby lattice, 
with the high symmetry points F, M and K. 



where 
and 

HsO=it2 ^ij^afScla^^j^^^ts Y ^a^a^^j ^ (3) 

on the ruby lattice shown in Fig. [TJ Here cj^/Qcr is the 
creation/annihilation operator of an electron on site i 
with spin a. As indicated in Fig. ma,b), t and ti are 
real first-neighbor hopping parameters, and ^2, ts are 
real second-neighbor hoppings (these appear with the 
imaginary number i in Eq.([3|) making the total second- 
neighbor hopping purely imaginary and time-reversal 
symmetric). The quantity Uij is equal to 1 if the elec- 
tron makes a left turn on the lattice links during the 
second- neighbor hopping, and is equal to -1 if the elec- 
tron make a right turn during that process. As is clear 
from FigdJ the unit cell of the lattice contains six sites so 
six two- fold degenerate bands will result. In addition to 
the real hopping parameters ti in (|2]), symmetry also al- 
lows complex, spin-dependent nearest neighbor hopping 
with imaginary components t' ^ as shown in Fig.[TJ We 
will discuss terms of this type later in Sec. IIVI 

The full Hamiltonian ^ can be diagonalized by going 
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FIG. 2. The energy bands without any spin-orbit coupling 
and with finite spin-orbit coupling, given by Eq.([2]). (a) Bulk 
energy bands along high symmetry directions in the absence 
of spin-orbit coupling. Note that there is a Dirac point at 
K for 1/6 and 2/3 filling, and a quadratic band touching 
point at F for 1/2 and 5/6 filling. (The underlying lattice 
is triangular, as it is for the honeycomb lattice.) Note also 
the flat bands along the F — M direction at 1/2 and 5/6 
filling, (b) The energy bands for finite spin-orbit coupling, 
with t2 = ts = O.lt. It is clearly seen that the Dirac points 
at the K points and quadratic band touching point at the F 
point are removed. 

to a momentum space representation, 

H = J2^i,Mk., (4) 

kcr 

where = (clka' 4ka' 4ka' 4ka' 4ka' 4ka) the six 

site basis, and i^ka is the Hamiltonian in /c-space. The 
bulk energy bands as well as the bands on a strip geome- 
try can be readily calculated. If the problem is solved on 
a two-dimensional strip, then periodic boundary condi- 
tions can be used in one direction (the direction parallel 
to the length of the strip). We diagonalize the Hamilto- 
nian matrix using standard LAPACK routines. 

The bulk energy bands and the energy bands on a strip 
geometry are shown in Figs. [2] and [3l respectively, for 
different sets of parameters. Having 6 sites per unit cell, 
our tight binding model yields 6 bands whose are doubly 
degenerate due to spin degrees of freedom. Thus the fill- 
ing factor is determined by counting how many of bands 
are fully occupied, which are 1/6, 1/3, 1/2, 2/3, 5/6. If 
one imagines shrinking the triangular plaquettes on the 
ruby lattice down to a point, the links will look like those 
of the honeycomb, showing that the underlying Br avals 
lattice is triangular (since the underlying Bravais lattice 
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FIG. 3. The energy bands on a strip geometry for the case 
that ti = t, t2 = ts = O.lt. At fining fractions 1/6 and 
2/3 edge mode cross the band gap an odd number of times, 
which clearly reveals the topological insulator phases. Their 
identification is also confirmed with a direct evaluation of the 
Z2 invariant. 



of the honeycomb is triangular). For the case ti = 
t2 = ts = shown in Fig. [2fa) there is no spin-orbit 
coupling on the lattice, and at all filling fractions the 
model predicts metallic behavior. Also in Fig. [2fa) we 
can see that when the energy is zero (1/2 filling), three 
bands are degenerate at the F point. For | filling and 
I filling, there are Dirac points at the K and points. 
Spin-orbit coupling can gap out the spectrum at some fill- 
ings [see FiglSfb)] and drive the model into an insulator. 
The nature of these insulators can be further understood 
by looking at the spectrum of the system with edges as 
shown in Fig.[3l We see that at | and | filling, there exist 
states at time reversal invariant points in k space, that 
is, at the points where kx = or = n/a. These states 
traverse the bulk band gap and are composed of an odd 
number of Kramers pairs, indicating that the insulating 
phase is in fact topological. ^^^^^ 

A direct evaluation of the Z2 invariant confirms this, 
as we will show in the next section. We found that the 
Dirac point at 2/3 filling in Fig. [3] is a linear crossing 
point, but for the special case of ti = ^2 = 0, and 
— 0.04t < ^3 < 0.04t, the absolute value of the slope 
dit kx = is less than 10~^. Therefore, in an experi- 
mental situation where temperature is finite (providing a 
low-energy cutt-off) it could be regarded as a quadratic 
crossing point. We also verified that a staggered poten- 
tial with a magnitude as small as O.Olt which breaks the 
C4 symmetry in the square plaquettes can enhance the 
slope by one order of magnitude. Our calculation is con- 
sistent with the discovery that extra symmetry of the 
underlying lattice may deform the shape of the Dirac 
node on the surface of 3D topological insulators giving 
rise to a warping effect^^ or a quadratic crossing in crys- 
talline topological insulators^^ The flat crossing at 2/3 
filling in our model may alter the low energy description 
of the edge modes and change their stability. In partic- 
ular, this may make them more susceptible to magnetic 
ordering from Couloumb interactions. 

Having established that our model supports the topo- 



logical insulator phase, we now turn to cases which ex- 
plore the whole parameter space of the model and deter- 
mine the phase diagrams at various filling fractions. 



III. PHASE DIAGRAMS OF THE MODEL 

We are interested in determining the phases of the 
Hamiltonian ([1]) as a function of filling fraction and 
the hopping parameters ^,^1,^2, and ts. From earlier 
^Q^]^62,63,66 70 kuow that we must consider three pos- 
sible phases: (i) Conductor, (ii) Insulator, (iii) Topolog- 
ical insulator. 

We determine the phase for a given set of parameters 
in the following way. First we must determine whether 
the system is conducting or insulating. After choosing 
a filling fraction, we search for the bottom of the "up- 
per band" with respect to this filling fraction and the 
top of the "lower band". For example, at 1/2 filling the 
"upper band" would be band 4 and the "lower band" 
would be band 3, with the labeling starting at the lowest 
energy band and the counting increasing as one moves 
to higher energy bands. Finding the extrema of a given 
energy band is a formidable task since the band struc- 
ture is rather complex and there are many local minima 
and maxima. We use an optimization algorithm called 
the differential evolution method. The method works 
well in most cases, but is not so efficient for some special 
parameters. In those cases, we use the software Mat he- 
mat ica to help us to determine the extreme values. If the 
system is insulating {i.e. there is a positive gap between 
the "upper band" minimum and the "lower band" max- 
imum), then we calculate the Z2 invariant according to 
the scheme proposed by Fu and Kane.^^ 

Taking the real-space triangular Bravais vectors of the 
ruby lattice as ai = ax and a.2 = f x + the recip- 

rocal lattice basis vectors are 



bi = — X - 

a 
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(5) 



Since our model possess inversion symmetry, we will cal- 
culate the eigenvalues of the parity operator at the four 
time reversal invariant points in the /c-space,^^ that is. 



ni 



bi 



where 



ni,n2 



n2, 



0,1. 



(6) 



(7) 



From the eigenvalues of the parity operator at the time 
reversal invariant momenta, the Z2 topological class can 
be determined as^^ 



(-ir = n<^a, 



(8) 



0=1 




where 

N 

= n ^2m(ra). (9) 

m—1 

Here u is the Z2 topological invariant, is one of the 
four time reversal invariant points defined as above and 
£,2m(^a) is the eigenvalue of the parity operator for the 
Bloch wave function of the 2m-th occupied band at the 
time reversal invariant point r^. 

Because of the large parameter space of the model, we 
must choose different "cuts" of the parameters to explore 
the phase diagrams. We will use the methods described 
above to determine the phases at filling fractions for dif- 
ferent hopping values. 

A. Phase diagrams for t2 = ts 

We begin by fixing the second neighbor hopping val- 
ues t2 = ts = Aso in ©. Thus we take uniform spin 
orbit coupling Aso and inter-triangle hopping ti as tun- 
ing parameters which are expected to drive the system 
into different phases. Fig. S] depicts the phase diagrams 
for filling fractions 1/6, 1/3,1/2,2/3, and 5/6. In the 
figure, and all related figures that follow in the paper 
the color coding is: Black= Conductor, Grey = Insulator, 
White= Topological Insulator. 

A variety of phases is seen at different fillings. Start- 
ing with I filling, we see that the middle axis Aso = is 
never in the topological insulator phase. This is easy to 
understand because in the absence of spin-orbit coupling, 
the system can only be a metal or trivial insulator. There 
also exists a horizontal straight line on the phase diagram 
which corresponds to the metallic phase when the inter- 
triangle hopping is zero. It is noteworthy that large areas 
of the phase diagram are occupied by the topological in- 
sulator. That is consistent with the observation of Dirac 
points at ^ filling for ^2 = ^3 = in the bulk energy 
bands and the crossing edge modes on the strip geome- 
try for Aso 7^ 0. The phenomenology is similar to the 
case of the decorated honeycomb lattice model. 

At I filling most areas are either a conductor or a 
trivial insulator, which is consistent with the lack of Dirac 
cones or quadratically touching points in the bulk energy 
bands when there is no spin-orbit coupling. However, 
as the spin-orbit coupling is turned on, at some special 
parameters the system can still be a topological insulator 
which are shown as white spots in Fig. IH^b). 

At I filling the phase diagram appears rather com- 
plex. In most areas it is a conductor, while for some 
areas the trivial insulator and the topological insulator 
can be distinguished. The topological insulator phase 
also appears on some narrow areas close to the region 
around Aso = ±0.7t and ti = —2M. 

The phase diagram for | filling consists mainly of the 
trivial insulator and the metallic phase. Nevertheless 
small regions around the center of the phase diagram oc- 
curring at small values of spin-orbit coupling present the 
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FIG. 4. Phase diagrams for different filling fractions. The 
figures (a)-(e) are for the filling fraction from 1/6 to 5/6, with 
filling fraction increasing in units of 1/6. For each filling, the 
vertical and horizontal axes measure ti and t2 — t2, — Aso, in 
units of t, respectively. Color coding is: Black=Conductor, 
Grey=Insulator, White=Topological Insulator. 

topological insulator phase. For much of the phase dia- 
gram, the conducting and trivial insulating phases occur 
alternately when the hopping strength and/or spin-orbit 
coupling are increased. The crossing states at /c = in 
the calculation on the strip geometry shown in Fig.[3l lies 
in the narrow region of the topological insulating phase. 

Finally, for | filling most of the area is occupied by 
the topological insulator, in strong contrast to the for- 
mer cases. This is also clearly seen from the calcula- 
tion in the strip geometry shown in Fig. [3] combined with 
the less stable quadratic band touching points^^^^^ (com- 
pared to Dirac points) that are clearly seen at 5/6 filling 
in Fig. [2fa) at the F point. 



B. Phase diagrams for fixed ti 

As we emphasized earlier, the model (|3j) supports 
two types of spin-orbit couplings, ^2,^3. In this sec- 
tion we investigate the interplay between theses couplings 
and the resulting phases. We describe phase diagrams 
with the same color code as before: Black=Conductor, 
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Grey=Insulator, White=Topological Insulator. We as- 
sume that the hopping between triangles, ti, is set to 
a fixed value, and the spin-orbit coupling t2 and ts 
are independently varied. This type of phase diagram 
takes a slice in the three dimensional parameter space 
{ti/t,t2/t^ts/t). Such phase diagrams reveal even more 
features than those presented earlier with t2 = ts (Fig|4|). 
In particular, we find that the two second- neighbor spin- 
orbit coupling terms can "compete" with each other and 
drive the system out of the topological insulator phase, 
even though each individually would place it there. 

We consider two cases: ti = and ti = t. The cor- 
responding phase diagrams are shown in Figs. [5]|6l The 
phase diagrams illustrated in Fig. [5] in which ti = are 
a special case. First note that in the absence of second- 
neighbor hopping it is clear the model must be in a trivial 
insulator phase since the only hopping is around triangu- 
lar plaquettes in the lattice. Thus, the origin at all filling 
fractions is a trivial insulator. However, away from the 
origin the behavior is rather different at different filling 
fractions. For example, close to the origin the model is in 
a trivial insulating state at 1/3 filling, while it remains a 
metal nearby at 2/3 filling (for the most part). Interest- 
ing, at I filling only the metallic phase appears. Also, at 
1/2 and 5/6 filling the phase diagram is mostly metallic. 

In Fig. [6] we show the phase diagrams for ti = t. Al- 
though those phase diagrams show a complex evolution 
with the hopping parameters, we can still draw some 
clear conclusions from them. For example, filling fraction 
1/6 and 5/6 are rich in regions of topological insulator, 
while filling fractions 1/3 and 2/3 are poor in regions of 
topological insulator. Filling fraction 1/2 tends to have 
roughly one-third to half of the parameter space occupied 
by the topological insulator phase. Ultimately, the expla- 
nation for these behaviors comes from the band structure 
in the presence of second-neighbor hopping. However, as 
shown in Fig. Efa), the bands at filling fraction 1/6 and 
2/3 both have a Dirac cone at the i^-point so the naive 
expectation might be for these two fillings to respond 
similarly when second-neighbor spin-orbit couplings are 
added. Our calculations clearly indicate this is not the 
case and the actual evolution is rather complicated. This 
feature should be borne in mind when viewing the present 
model as the result of a self-consistent Hamiltonian in 
which spin-orbit coupling was spontaneously generated 
from interactions i^^i^"*^" — We note that if the signs of any 
two among ^1,^2, and ts are reversed, the figures show the 
phase of the system is unchanged. For example, if one 
fiips the phase diagram for ti = t (See Fig. [6]) around 
the axis t2 = or ts = 0, the phase diagram obtained 
is identical to the phase diagram for the case ti = —t. 
This symmetry is easy to verify from the Hamiltonian 
([1]). Indeed, the sign of ti can be absorbed via a simple 
gauge transformation of electron operators. The latter is 
defined as follows: c^(c) — c^(— c) for up triangles and 
c^{c) for down triangles. This transformation 

fiips the signs of ti and ^3. Thus this simple argument 
implies that the phase diagram is symmetric via such 
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FIG. 5. Phase diagrams for ti =0. The horizontal axis is 
t2/t, and the vertical axis is ts/t. The figures (a)-(e) are for 
the filling fraction from 1/6 to 5/6, increasing in units of 1/6. 
For all fillings the origin t2 = ts = is a trivial insulator. 



transformations. Moreover, if the signs of t2 and ts are 
fiipped, the phase diagram will not change, which can be 
simply understood by noting that this fiipping can also 
be absorbed into the magnetic fields for different spin 
species, i.e. Uij —i^ij- 

We also studied other cases with different values of 
inter-triangle hopping. For example, when ti = 2t and 
the filling fraction is |, if ^2 = ^ and ^3 = 0, or ^2 = 
and ts = 0.4t, the system is topological insulating, while 
if t2 = t and ts = 0.4t, the system is a metal. A possible 
explanation for such phenomena is the inner magnetic 
field created by the two different types of spin-orbit cou- 
phng may have opposite contribution to the phase of the 
hopping terms, and therefore may cancel each other for 
some parameter values. A similar inverse case also exists 
where a single type of spin-orbit coupling does not re- 
sult in the topological insulator phase while if both types 
of spin-orbit coupling are present, the system becomes a 
topological insulator. 
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FIG. 6. Phase diagrams for ti — t. The horizontal axis is 
t2/t, and the vertical axis is ta/t. The figures (a)-(e) are for 
the filling fraction from 1/6 to 5/6, increasing in units of 1/6. 



IV. FLAT BAND FRACTIONAL QUANTUM 
HALL EFFECT 

In recent years, lattice models with flat bands have 
attracted attention for a number of reasons, among them 
are enhanced interaction effects. ^^^^ Since the scale of 
the kinetic energy is set by the band width, if the band 
width vanishes, any residual inter-particle interactions 
will be "large" and may drive the system into a strongly 
correlated state such as a Wigner crystal^^ or a fractional 
quantum Hall state^^ 

Flat band lattice models in which the flat bands pos- 
sess a finite Chern number are thought to be excel- 
lent candidates for systems that might realize a frac- 
tional quantum Hall effect when the flat band is par- 
tially filled. ^^^^ Indeed, exact diagonalization studies on 
the checkerboard lattice^i^ and other lattice^*^ sup- 
port the development of a fractional quantum Hall state 
at certain filling fractions. 

In order to maximize the effectiveness of the interac- 
tions in driving a fractional quantum Hall state, the flat 
band should be as flat as possible and the band gap to the 
next band should be as large as possible. (Virtual tran- 
sitions to higher lying bands with finite Chern numbers 




FIG. 7. Phase diagrams for ti = 2t. The horizontal axis is 
t2/t, and the vertical axis is ta/t. The figures (a)-(e) are for 
the filling fraction from 1/6 to 5/6, increasing in units of 1/6. 



are more effective at disrupting the fractional quantum 
Hall state than those with zero Chern number.) If we call 
the bandwidth of the flat band W and the energy differ- 
ence between the lowest point of the band above the flat 
band and the highest point of the flat band Eg , the figure 
of merit is Eg/W ^ and this should be much larger than 
unity. If the characteristic strength of the inter-particle 
interactions is [/, the regime W <^U <^ Eg will involve 
little mixing from the higher bands and the physics will 
be dominated by the interactions in the flat band with a 
finite Chern number. 

In this section, we show that the ruby lattice possess 
flat bands with finite Chern number. One can readily 
reach the regime Eg/W ~ 70. 

We first study a new model without the spin-orbit cou- 
pling in Eq. (j3]) , but with the imaginary part on the hop- 
ping terms Eq.([2j). Specifically, in the Hamiltonian (|2]) 
the hopping terms are substituted by 

t' = t ^ idzti, 

t[ =tir^i(T,tu, (10) 

The complex hopping parameters above can be artifi- 
cially synthesized in fermionic cold atomic optical lat- 
tices via a specific tuning of Raman fields, ^^^^ or they 
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FIG. 8. The energy bands with U = 1.2t,ti^ = -1.2t,tii = 
2.6t,t4r = -1.2t. 



can arise in some spin-orbital perovskites where strong 
intrinsic spin-orbit coupHng can make the hopping com- 
plex and spin-dependent.^^ 

To explicitly break time-reversal symmetry, we assume 
the fermions are spin polarized and calculate the band 
structure and Chern number for one spin orientation. 
The Chern number of the n-th band is defined as^- 

Cn = ^ [ d^kFi2{k), (11) 

^TT Jbz 

where the integral is taken for the two-dimensional Bril- 
louin zone, and Fi2{k) is the associated field strength 
defined as 

FMk) = ^Mk)-^Mk), (12) 

where = — ^(iink|^f-|'^nk) is the Berry connection 

and I^Xnk) are the Bloch wavefunctions of the n^^ band. 
The full Chern number is the sum of Cn over all occupied 
bands, which in our case is just the single lowest energy 
band. The calculation of Chern numbers is based on the 
methods in Ref.[^, where Fi2{k) is expressed in some 
U{1) gauge transformation link variables, and then the 
integration is converted into a summation in the /c-space. 
With only the hopping parameters (p!Q|) our flattest band 
with finite Chern number had W/Eg ^ 13, which is not 
very large. 

To obtain a nearly flat band with nonzero Chern 
number, we needed to add more terms to the Hamil- 
tonian. One option is to add the hopping inside the 
square in the diagonal directions, which is called t^r- 



This is shown schematically in Fig. [B^c). The nonzero 
Chern numbers and nearly flat bands do occur in this 
case. For example, with U = 1.2t,ti^ = —1.2t^tii = 
2.6t,t4r = —1.2t the Chern number is -1 and the gap = 
2.398t, and band width = 0.037t, which gives W/Eg ^ 
70. The corresponding band structure is shown in Fig. [51 
Based on the results of exact diagonalization studies on 
other lattices^^^!^ and the general arguments given 
before, ^^^^ we expect a flat band fractional quantum 
Hall effect to be realized on the ruby lattice as well for the 
appropriate fractional filling of the flat band. This could 
form the basis of a lattice model of a fractional topolog- 
ical insulator by taking two time-reversed copies. 
V. CONCLUSION 

In conclusion, we have studied the energy bands of 
a tight-binding model on the ruby lattice, and also ob- 
tained the phase diagrams for a variety of filling fractions. 
The phase diagram is rather complex in its dependence 
on the various types of spin-orbit coupling and nearest- 
neighbor hopping we considered. We have seen how var- 
ious spin-orbit coupling terms can "compete" with each 
other and also "support" each other. 

In a related spinless model we calculated the Chern 
number for the nearly flat bands, and showed that they 
will likely support a robust fractional quantum Hall effect 
with W/Eg ^ 70. This could lead to possible lattice 
models of a fractional topological insulator. 

Since the ruby lattice has earlier appeared in the study 
of topological spin models, ^^^^^ the current work opens 
the way for an exploration of the transitions between 
various interesting phases in a more general interact- 
ing model. Current technology in cold atomic gases 
should allow an experimental study^"— ^ (although per- 
haps with challenges) of fermions in the ruby lattice and 
will likely raise new questions to challenge theory. 
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